Add WebGPU mul and sigmoid test coverage#20504
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20504
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit f1a70a6 with merge base af62dce ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @iamorlando! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
@pytorchbot label "release notes: ops & kernels" |
pytorch-bot
Bot
added
the
release notes: ops & kernels
meta-cla
Bot
added
the
CLA Signed
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
There was a problem hiding this comment.
Pull request overview
Adds WebGPU backend runtime support for aten.sigmoid.default, expanding the set of transformer/LLM-oriented ops that can execute via the existing Vulkan-delegate serialization consumed by the WebGPU runtime.
Changes:
- Adds an fp32 WGSL sigmoid compute shader plus a generated embedded WGSL header.
- Registers a new C++ WebGPU runtime implementation for
aten.sigmoid.defaultand wires it into the WebGPU CMake build. - Extends WebGPU Python test infrastructure (allowlist + op-test registry/cases) and updates WebGPU docs/support matrices to include sigmoid.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| backends/webgpu/TODO.md | Updates current state / next-steps notes to reflect broader op coverage including sigmoid. |
| backends/webgpu/test/tester.py | Adds aten.sigmoid.default to the VulkanPartitioner allowlist used by WebGPU tests/export harness. |
| backends/webgpu/test/TARGETS | Adds a Buck python_unittest target for the new sigmoid export smoke tests. |
| backends/webgpu/test/ops/sigmoid/test_sigmoid.py | New export-level smoke tests for sigmoid (including chained sigmoid) plus deterministic input generators used by op-test cases. |
| backends/webgpu/test/ops/sigmoid/init.py | New package marker for sigmoid test module. |
| backends/webgpu/test/op_tests/test_schema.py | Extends schema assertions to require registered sigmoid op-test suites/cases and golden dtype expectations. |
| backends/webgpu/test/op_tests/test_generator.py | Ensures sigmoid cases are included in the “every case delegates” contract check. |
| backends/webgpu/test/op_tests/generate_op_tests.py | Updates generator CLI docstring to include sigmoid in --ops examples. |
| backends/webgpu/test/op_tests/cases.py | Registers a new sigmoid op-test suite with multiple shape/range/chained cases. |
| backends/webgpu/runtime/ops/sigmoid/sigmoid.wgsl | New fp32 sigmoid kernel WGSL. |
| backends/webgpu/runtime/ops/sigmoid/Sigmoid.cpp | New runtime operator implementation + dispatch setup for aten.sigmoid.default. |
| backends/webgpu/runtime/ops/sigmoid/sigmoid_wgsl.h | Generated embedded WGSL header for the sigmoid shader + workgroup constants. |
| backends/webgpu/README.md | Updates status/progress and the operator support matrix to list sigmoid (and related already-landed ops). |
| backends/webgpu/CMakeLists.txt | Adds runtime/ops/sigmoid/Sigmoid.cpp to the WebGPU backend build sources. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@claude review this PR |
|
Claude Code is working… I'll analyze this and get back to you. |
|
@claude review this PR |
|
Claude Code is working… I'll analyze this and get back to you. |
|
Hey @ianbarber, thanks for this PR! It looks like main may already cover most of this. |
Hi @JCNTH !
|
Hey @iamorlando more in-depth test coverage would be great! |
| `SigmoidModule`, `N`, and `_det_input` are imported by `cases.py` to drive the | ||
| declarative op-test suite. `SigmoidTest` is the export-delegation | ||
| smoke test. Sigmoid is on the Llama critical path (`F.silu` -> `sigmoid` + `mul`); the | ||
| deterministic input spans the saturation tails. | ||
| deterministic inputs span the saturation tails. |
| def test_add_mul_sigmoid_rms_norm_registered(): | ||
| from executorch.backends.webgpu.test.op_tests import cases # registers |
| @register_op_test("sigmoid") | ||
| def _sigmoid_suite() -> WebGPUTestSuite: | ||
| # sigmoid has no CONFIGS table; cover unary shapes directly (tol 1e-4). | ||
| return WebGPUTestSuite( | ||
| module_factory=lambda: SigmoidModule(), | ||
| module_factory=lambda variant="regular": ( | ||
| SigmoidChainedModule() if variant == "chained" else SigmoidModule() | ||
| ), |
|
@claude review |
|
Claude Code is working… I'll analyze this and get back to you. |
| module_factory=lambda variant="regular": ( | ||
| SigmoidChainedModule() if variant == "chained" else SigmoidModule() | ||
| ), |
iamorlando
changed the title
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
backends/webgpu/test/op_tests/test_generator.py:85
test_every_case_delegatesis meant to prevent silent CPU fallback, but the current assertion (_has_vulkan_delegate) only proves that at least one delegate exists. With multi-op cases like the new chained sigmoid, this can still pass even if some target ops remain as CPU nodes. Consider additionally asserting that the expected op is absent from the lowered Edge graph for ops where full delegation is required (e.g.,mul.Tensorandsigmoid.default).
def test_every_case_delegates():
# Contract: every registered case must lower to a VulkanBackend delegate. An op that
# silently CPU-falls-back would otherwise produce a misleading golden-equals-golden pass.
for op in ("add", "mul", "sigmoid", "rms_norm"):
suite = op_test_registry[op]
for case in suite.cases:
_module, _inputs, prog = g.export_case(suite, case)
assert g._has_vulkan_delegate(prog), f"{op}/{case.name} did not delegate"
| def test_export_delegates(self) -> None: | ||
| et = _export(SigmoidModule().eval(), _det_input()) | ||
| found = any( | ||
| d.id == "VulkanBackend" | ||
| for plan in et.executorch_program.execution_plan | ||
| for d in plan.delegates | ||
| ) | ||
| self.assertTrue(found, "Expected a VulkanBackend delegate (sigmoid)") | ||
| for name, module, x in ( | ||
| ("regular", SigmoidModule().eval(), _det_input()), | ||
| ("chained", SigmoidChainedModule().eval(), _det_input()), | ||
| ): | ||
| with self.subTest(name=name): | ||
| et = _export(module, x) | ||
| found = any( | ||
| d.id == "VulkanBackend" | ||
| for plan in et.executorch_program.execution_plan | ||
| for d in plan.delegates | ||
| ) | ||
| self.assertTrue(found, f"Expected a VulkanBackend delegate ({name})") |
| def _lower(m: torch.nn.Module, x: torch.Tensor): | ||
| ep = torch.export.export(m, (x,)) | ||
| return to_edge_transform_and_lower( | ||
| ep, partitioner=[VulkanPartitioner()] | ||
| ).to_executorch() | ||
| return to_edge_transform_and_lower(ep, partitioner=[VulkanPartitioner()]) | ||
|
|
| # Contract: every registered case must lower to a VulkanBackend delegate. An op that | ||
| # silently CPU-falls-back would otherwise produce a misleading golden-equals-golden pass. | ||
| for op in ("add", "rms_norm"): | ||
| for op in ("add", "mul", "sigmoid", "rms_norm"): | ||
| suite = op_test_registry[op] | ||
| for case in suite.cases: |
Summary
Adds WebGPU test coverage for existing
aten.mul.Tensorandaten.sigmoid.defaultsupport.This updates the WebGPU Python op tests to cover:
mulbroadcast shapes, including leading-dimension and mixed 4D broadcast casesmulandsigmoidmulandsigmoidcases must lower to the WebGPU delegate pathNo WebGPU runtime, shader, CMake, README, or TODO files are changed in this PR.
Test plan
git diff --check.venv-pyproject/bin/python -m py_compile backends/webgpu/test/op_tests/cases.py backends/webgpu/test/op_tests/test_generator.py backends/webgpu/test/op_tests/test_schema.py backends/webgpu/test/ops/test_mul.py backends/webgpu/test/ops/test_sigmoid.pyPATH=/Users/orlando/Documents/dev/executorch/.venv-pyproject/bin:$PATH .venv-pyproject/bin/lintrunner --data-path /tmp/executorch_lintrunner_pyproject backends/webgpu/test/op_tests/cases.py backends/webgpu/test/op_tests/test_generator.py backends/webgpu/test/op_tests/test_schema.py backends/webgpu/test/ops/test_mul.py backends/webgpu/test/ops/test_sigmoid.pyPYTHONPYCACHEPREFIX=/tmp/executorch_pyproject_pycache .venv-pyproject/bin/python -m pytest backends/webgpu/test/ops/test_mul.py backends/webgpu/test/ops/test_sigmoid.py backends/webgpu/test/op_tests/test_schema.py backends/webgpu/test/op_tests/test_generator.py -vvResult:
13 passed in 120.77s